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Information  in  Sensor  Fusion 


Combining  information  from  two  or  more 
sensors 

Combining  information  from  different  modes  of 
single  sensor 

Fuse  information  from  different  algorithms 


Motivations 


Limits  of  performance  for  single  sensors 
can  be  pushed  only  in  small  increments 

Multisensor  integration  MAY  LEAD  TO 
improved  detection  and  identification  with 
significantly  lower  false  alarms 

More  data  -  potential  for  improved 
information  from  measurements 


Multisensor  Integration 

Sensors  designed  to  operate  independently: 
integration  is  an  “after  thought” 

Fusion  approaches  have  been  system  and  function 
specific 

Conventional  techniques,  models  assume  identical 
sensor  statistics,  equal  thresholds,  high  SNRs,  and 
uncorrelated  sensor  noise  fields  -  example  Radar 
PDI 

Issues  remain  in  integrating  dissimilar  sensors 
collecting  data  asynchronously  and  communicating 
to  central  processor  with  different  time  delays 


18th  Century  Information  Fusion 


■  1786:  Condorcet  -  Democracy  Models: 

Each  individual  has  probability  p  of  making  correct 
decision: 

What  is  the  probability  of  democracy  making  the  correct 
decision  ? 

Democracy  Model 

p:  individual  probability  of  making  correct  decision; 

n:  number  of  members  of  democracy 

P_n:  probability  of  democracy  making  correct  decision? 

If  p  >  Vi  then  P_n  >  p  >  Vi  P_n  approaches  1  as  n  grows 

p  <  Vi  then  P_n  <  p  <  Vi  P_n  approaches  0  as  n  grows 

p=  1/2  then  P_n  =  p=  1/2 

Informally,  democracy  will  do  well 
if  p  >1/2  and  will  do  bad  if  p<l/2 


Information  Fusion  in  Twentieth  Century 

1956,  Reliability:  Von  Neumann  showed  how  to  build 
a  reliable  system  using  unreliable  components 
under  independent  failures. 

1962,  Pattern  Recognition:  Chow  showed  optimal 
threshold  fuser  for  multiple  independent  classifiers. 

1969,  Forecasting:  Bates  and  Granger,  “better” 
forecasts  can  be  made  by  combining  different 
forecast  methods  rather  than  picking  one  of  them 

Importance  of  “fusing”  rather  than  picking  the  “best” 
has  been  demonstrated  in  a  number  of  disparate 
disciplines 


Information  Fusion  in  Late  Twentieth  Century 


Advances  in  Computing  and  Complex  Engineering  systems 
posed  new  challenges: 

-  Distributed  Detection:  Bayesian  methods  for  object  detection  using 
measurements  from  different  detectors 

-  Sensor  Fusion:  Multiple  sensors  became  essential  to  many 
engineering  systems  -  fusion  is  part  of  the  problem  specification 

-  Mixture  of  Experts:  Function  and  regression  estimation  can  benefit  by 
combining  multiple  estimators 

-  Multiple  Classifiers:  There  is  no  single  best  classifier  but  “combined” 
one  is  better  than  its  components 

Information  Fusion  began  taking  roots  as  a  discipline  unto 
itself: 

Office  of  Naval  Research  sponsored  first  workshop  on  Information 
Fusion  in  1996,  jointly  with  National  Science  Foundation  and 
Department  of  Energy 


First  Workshop  on  Information  Fusion 

Office  of  Naval  Research  was  the  lead  sponsor,  together  with 
National  Science  Foundation  and  Department  of  Energy 


Brought  together  scientists  from: 
Engineering,  Computer 
Science,  Mathematics, 
Econometrics, 
Bioinformatics,  Statistics 


PROCEEDINGS 


WORKSHOP  ON  FOUNDATIONS  OF 

Information  /  Decision  Fusion 

WITH  APPLICATIONS  TO  ENGINEERING  PROBLEMS 


This  workshop  launched  the  field 
of  Information  Fusion 
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Information  Fusion  Area  Today 

Integral  part  of  newer  disciplines  including: 

•  Distributed  Sensor  Networks 

•  Cyber  Data  Mining 

•  Cognitive  sensor  Fusion 

Dedicated  International  Conferences: 

1.  International  Conference  on  Information  Fusion  (13th  in  Edinburgh, 
2010) 

2.  International  Conference  on  Multisensor  Fusion  and  Integration 
(Salt  Lake  City,  UT,  2010) 

Journals: 

Information  Fusion  (2000) 

International  Journal  of  Distributed  Sensor  Networks  (2005) 

Journal  of  Advances  in  Information  Fusion  (2006) 


Information  Fusion  area  -  last 

decade  or  two? 

Rich  Information  Sources 

-  Sophisticated  sensors  -  visual,  hyperspectral,  radiation,  chemical, 
biological,  and  others 

-  Information  sensors  -  web  crawlers,  information  servers, 
sophisticated  databases 

Advances  in  Computation 

-  Fusion  problems  have  become  complex 

-  Powerful  computer  hardware  and  algorithms  have  been  developed 

Advances  in  Networking 

-  Made  access  to  computing  and  data  resource  easier 

-  Wireless  networking  made  ad  hoc  deployments  possible 

-  High-performance  networks  made  it  possible  to  support  large  data 
transport  and  remote  control  possible 


NEW  in  September  2010 


IEEE  2010  INTERNATIONAL  CONFERENCE  ON  MULTISENSOR 
FUSION  AND  INTEGRATION  FOR  INTELLIGENT  SYSTEMS  (IEEE  MFI 
2010),  SEPTEMBER  5-7,  2010 

The  theme  of  IEEE  MFI  2010  was  Cognitive  Sensor  Fusion 


Here  the  goal  of  multi-sensor  fusion  systems  is  to  achieve  human-like 
performance  in  terms  of  perception,  knowledge  extraction,  and  situation 
assessment,  exploiting  symbolic  and/or  dynamical  systems  approaches. 
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Motivation 

Detection  of  Low-level  Radiation  Sources 


Task: 

•  Detect  the  sources  based  on 
sensor  measurements 


Several  underlying  math  problems 
related  to  detection  networks  are  open. 
Our  work 

-addresses  network- based  detection 
-provides  answers  using  statistical 
estimation  and  packing  numbers 


Difficulty  of  Detecting  Low-level  Radiation  Sources 

The  radiation  levels  are  only  slightly  above  the  background  levels 
and  may  appear  to  be  "normal"  background  variations 

•  Varied  Background:  Depends  on  local  natural  and  man-made 
sources  and  may  vary  from  area  to  area 

•  Probabilistic  Measurements:  Radiation  measurements  are 
inherently  random  due  to  underlying  physical  process  - 
gamma  radiation  measurements  follow  Poisson  Process 

Several  solutions  are  based  on  thresholding  sensor  measurements 


Well-Studied  Problem:  Has  been  studied  for  decades  using  single  or 
co-located  sensors:  analytical,  experimental  and 

-  sensor  networks  offer  "newer"  solutions  but  also  questions 

Open  Mathematical  Question:  Q1 

I  s  there  a  mathematical  quantification  for  a  network  of  sensors  to 
achieve  better  performance  than  single-sensor  detectors? 


Detection  of  Sources  amidst  Background  Noise 

A  Traditional  Method  for  Detection: 

1.  SPRT  to  infer  detection  from  measurements  at  sensors; 

2.  Fuse  the  Boolean  decisions  at  fusion  center. 


Specific  Question:  Are  there  methods  that  perform  better? 

Generic  Question:  Are  there  classes  of  detection  problems  that  benefit 
from  “fusing"  measurements  in  place  of  decisions  ? 

Our  Results:  Answer  is  yes  to  both  questions  under 

1.  Lipschitz  smoothness  conditions  -  limited  shielding  conditions 

2.  Vapnik-Chenvenenkis  conditions  -  discrete  intensity  drops 

3.  *  SPRT-sequential  probability  ratio  test 


Detection  Using  Localization  fl  * 

s 


Proposed  Method  for  Detection: 

1.  Estimate  the  source  parameters  using  measurements  -  As;(xS9 ys) 

2.  Utilize  likelihood  ratio  test  □  ~  at  the  fusion  center 
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Explanation  of  Results 


A  fixed-threshold  SPRT  detection  method  optimizes  the  detection 
performance  within  a  certain  neighborhood  of  state-space. 

-  characterized  by  sets  S  ,  S 

rL  th 


Localization  facilitates  the  adaptation  of  the  threshold  to  estimated 
neighborhood  of  the  state-space  albeit  with  a  certain  error  probability. 

Our  method  achieves  a  trading-off  between 

•error  probability  of  the  localization  method  in  estimating  the 
neighborhood  and  needed  SPRT  thresholds;  and 
•probability  of  "uncovered"  regions  of  fixed- threshold  SPRT 
By  suitable  trade-off  one  can  exceed  the  performance  of  the  latter. 


Summary  of  Results 

I  mproved  detection  using  measurements  at  fusion  center  compared  to 
existing  decision  fusion  methods,  using  robust  localization,  under: 

General  non-smooth  conditions: 

Separability  of  probability  ratios 

-  complex  analysis  and  less  intuitive  conditions 

+  valid  under  complex  shielding  of  radiation  sources 
Smoothness  conditions: 

Lipschitz  separable  probability  ratios;  and 
Lipschitz  source  intensity 

+  intuitive  conditions:  "bigger"  parameter  space  is  better 

-  valid  typically  under  open-space  environments 

First  mathematical  proofs  for  this  class  of  problems  to  show: 

i)  a  network  of  sensors  performs  better  than  single  or  co-located  sensors 

ii)  measurement  "fusion"  performs  better  than  detection  fusion 
Performance  improvement  is  characterized  by  the| packing  number  j 

single  scalar 


Summary 


We  proposed  source  detection  method  using  a  network  of  sensors: 

-  utilizes  localization  followed  by  SPRT 

-  out-performs  :  under  both  smooth  and  non-smooth  conditions 

any  single  SPRT  method; 
majority  and  other  fusion  methods 
•For  radiation  point  source  detection: 

-  performs  better  than  existing  decision  fusion  methods 
•Shows  cyber- physical  trade-off:  better  detection  at  higher  cyber 
cost 

•lowest  cost:  single  sensor  with  SPRT 
•intermediate  cost:  SPRT  at  sensors  and  Boolean  fusion 
•highest  cyber  cost:  send  measurements  and  localization- based 
fusion 


Cognitive  Radar 


The  University  of  Arizona  is  developing: 

-  Robust  Bayesian  channel  models  for  target  recognition  and  surveillance 

-  Algorithms  for  waveform  optimization  derived  from  the  Bayesian  models 

Cognitive  Radar: 

-  An  approach  to  radar  that  closes  the  loop  between  exploitation  (signal  processing) 
and  control/opfimi7ation  ofthe  measurements 

-  Interrogate  the  radar  environment  through  smart  control  of  the  interrogation 
. p  rope  rties  (I:  e'v"  beamsteefinig;"  PRF;  pul'Se"  shape)'  "=>"  opt]  ml  zeavai  fa  bl  e  1 1  rrie/en  ergy 

-  Adaptive  measurement  control 

*  Where  to  go,  what  to  transmit,  where 
to  aim,  who  to  cooperate  with,  ... 


Inte  110 g at e/ ELbirtmiaie  the  Channel 


Compute  Waveforms 
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Feed  Understanding  Back 
to  Transmitter 


Priorities  and 
Constraints 

Additional 

Knowledge 

Sources 

Fixed  search  pattern: 
Includes  areas  where 
target  unlikely 


Adaptive  search  pattern: 
Focus  radar  on  important, 
but  uncertain,  areas 


Use  past  observations  to  enhance  future  measurements 


THE  UNIVERSITY  OF  ARIZONA, 


Enor  Rale  00  Transmissions) 


Sample  Results:  Two  Applications 


Target  ID 


*  Characterize  targets  by  transfer  function 
and  compute  variance  over  the  classes: 
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Search  &  Track 


Target  Parameters 
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Describe  target  parameter  space  as  a  grid  of 
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-  A  measure  of  entropy  vs.  frequency 
parameterized  by  target  classes 


wf  CIUHef 
-  -  - ■  M I  wto  Clutter 
SNR  wYCkifler 
Itt  rfe  band 


Gain  due  to 
adaptive, 
optimized 
waveforms 


target  probabilities 

Convertthe  probabilistic  representation  into 
the  best  beam  steering  location 

Probability  of 
detecting  a  weak 
target  moving 
through  search 
zone; 

(500-target  test) 
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Ongoing  Work 

At  UA: 

-  Computationally  efficient  and  robust  probability  update  procedures 

•  For  example,  how  do  we  perform  stable  updates  of  the  Bayesian  probability  map  when 
interference/clutter  have  unknown  pdf? 

-  Probability  updating  procedures  for  multiple  platforms  (mapping  of  target  parameters 
to  range/Doppler/angle  is  unique  for  each  platform) 

-  Adaptive  PRF  selection  for  range-Doppler  ambiguity  mitigation 

-  Practical  classification  algorithms  and  waveform  constraints  (e.g.,  constant  modulus) 

Elsewhere: 

-  Dr.  Dan  Fuhrmann  (now  at  Michigan  Tech  University) 

•  “Active-testing  surveillance  systems,  or  playing  20  questions  with  a  radar”;  2003  ASAP 
workshop 

-  Proposes  radar  measurement  optimization  via  probabilistic  representation  (closed  loop  system!) 

-  Dr.  Simon  Haykin,  McMaster  University 

•  “Cognitive  Radar:  A  Way  of  the  Future”,  IEEE  Signal  Processing  Magazine,  Jan.  2006. 

-  Summarizes  philosophy  of  cognitive  radar  and  feedback  from  receiver  to  transmitter 

•  Cognitive  Tracking  Radar;  Cognitive  Dynamic  Systems 

(http://soma.mcmaster.ca/havkin.php) 

-  Dr.  Arye  Nehorai,  Washington  University-St.  Louis 

•  Adaptive  waveform  parameters  and  radar  flight  path  (Asilomar  2008) 

•  "Adaptive  polarized  waveform  design  for  target  tracking  based  on  sequential  Bayesian 
inference,"  IEEE  Trans,  on  Signal  Processing,  Mar.  2008. 


Signal  Processing  for 
Networked  Sensing 

A  majority  of  performance  improvements  in 
Sensors,  Networking  and  Communication 
connectivity  are  expected  to  come  from 
conceptualization  of  new  systems,  and  through 
innovations  in  Signal  Processing. 

Leverage  developments  in  Signal  Processing 
techniques  to  bring  about  improvements  in 
sensing,  target  resolution,  small  target 
detectability,  multi-target  tracking,  and 
address  hard  problems  in  sensor  data  fusion, 
track  fusion  and  communications. 


